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ABSTRACT 

Objectives: To evaluate the Nordic Patient 
Experiences Questionnaire (NORPEQ) for data quality, 
reliability and validity following surveys of patients in 
Finland, Norway, Sweden and the Faroe Islands. 
Design, mettiods and participants: The NQRPEQ 
was mailed to 500 patients randomly selected after 
receiving inpatient treatment in Finland, Norway and 
Sweden. The NORPEQ was also included in a national 
survey in Norway and in the Faroe Islands. 
Dimensionality was assessed using principal 
component analysis and internal consistency by item- 
total correlation and Cronbach's a. Construct validity 
was assessed by correlating NORPEQ scores with 
variables known to be related to patient experiences. 
Setting: Somatic hospitals in Finland, Faroe Islands, 
Norway and Sweden. 

Primary and secondary outcome measures: Item 
missing, internal consistency reliability and construct 
validity. 

Results: Response rates ranged from 45.8% in 
Norway to 84% for Sweden. Levels of missing data 
were low for all items across the surveys. Principal 
component analysis identified one component with six 
experiences items. Mean NORPEQ scores ranged from 
74 to 79 on the 0-1 00 scale, where 1 00 represents the 
best possible experiences. Cronbach's a ranged from 
0.84 in Finland to 0.88 in Sweden. 
Conclusions: The NORPEQ is a brief measure of 
patient experiences that covers important aspects 
of the healthcare encounter. It shows good evidence of 
reliability and validity. 

Practice implications: The NORPEQ instrument is 
recommended for cross-national comparisons of 
healthcare experiences for the four Nordic countries. 
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INTRODUCTION 

The literature relating to the development 
and testing of healthcare quality indicators 
that assess patient experiences and satisfac- 
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ARTICLE SUMMARY 



Article focus ^ 

■ The NORPEQ was designed to include a core set 
of questions covering the most important 
aspects of patient experiences that can be used 
cross-nationally and alongside existing longer- 
form national survey questionnaires. 

■ The aim of this study was to evaluate the 
psychometric properties of NORPEQ in four 
Nordic countries. 

Key messages 

■ On the basis of a rigorous process of question- 
naire development and evaluation including 
forward-backwards translation, levels of 
missing data analysis, dimensionality, internal 
and construct validity, the NORPEQ shows good 
evidence of reliability and validity. 
The NORPEQ instrument is recommended for 
cross-national comparisons of healthcare expe- 
riences in the Nordic countries. ^ 

Strengths and limitations of this study 

■ The NORPEQ includes what are evaluated to be 
the most important aspects of experiences for 
patients. Levels of missing data were generally 
very low across countries indicating the accept- 
ability of the questionnaire. 

■ The NORPEQ was tested in four countries, and 
there is good evidence for the cross-cultural 
equivalence of the questionnaire. 

■ Results were based on pilot surveys in two 
countries and should be further evaluated 
following national surveys. Psychometric prop- 
erties of the NORPEQ should also be tested in 
Denmark and Iceland. ^ 



tion is extensive. Such indicators have 
gained increasing importance following the 
work of international organisations, such as 
the Organisation for Economic Cooperation 
and Development and the WHO, which have 
emphasised the importance of the patient's 
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perspective by capturing patients' experiences or satis- 
faction in the evaluation of the quality of healthcare 
delivery.^ Much of this work has been at the local level 
in relation to local providers, but national governments 
require comparisons of providers.^ ^ 

All the Nordic countries have a history of measuring 
patient experiences and patient satisfaction. The patient 
groups surveyed, survey methodology and questionnaire 
content have however differed. In Finland, the National 
Institute for Health and Welfare (THL) questionnaire 
has been used in different surveys,'' but there has not 
been a national patient experiences survey. In Norway, 
national patient experience surveys have included 
a variety of patient groups, including somatic inpatients,'' 
outpatients,*'^^ psychiatric inpatients and otitpatients,' 
and parents of paediatric patients.^ In Denmark, 
national patient experience surveys have been 
conducted for many years, both among psychiatric and 
somatic patients.^ In Sweden, smaller scale studies have 
been conducted with the Picker Patient Experience 
Questionnaire,^** and since 2009, the Swedish Associa- 
tion of Local Authorities and Regions in Sweden has 
conducted regular national surveys of users of primary 
care services and among somatic and psychiatric inpa- 
tients and outpatients. The Faroe Islands national 
patient experience surveys were conducted in 2007 and 
2010. Iceland has conducted national surveys of health- 
care providers in 2002 and 2005.^ 

The Nordic Council of Ministers initiated a cross- 
national collaboration to develop a set of quality indica- 
tors, including patient experiences, for measuring the 
quality of the health services across the Nordic countries. 
The work follows the Organisadon for Economic Coop- 
eration and Development's conceptual framework in 
which one of the core healthcare quality indicators is 
patient-centeredness.^^ The similarities of the healthcare 
systems within the Nordic countries make them well 
suited for cross-national comparisons; care is virtually free 
at the point of use, governments use monetary incentives 
and various directives to influence the priorities of service 
producers and accountability for service provision is 
delegated to local authorities.^^ During the past decade, 
the Nordic countries have increasingly presented infor- 
mation regarding the performance of healthcare 
providers. Public reports and internet sites such as the 
'Free Hospital Choice' in Norway (http://www.fritLsyke- 
husvalg.no) and the 'National Patient Questionnaire' in 
Sweden (http://www.skl.se/nationellpatientenkat) have 
published results of surveys of patient experiences, the 
intentions being quality improvement and public 
accountability. Such information can also provide a basis 
for cross-national comparisons.^ Cross-national 
comparisons of patient experiences in the Nordic coun- 
tries provide information about quality of healthcare, 
which may inform the Nordic citizens, patients, politicians 
and leaders and healthcare personnel thereby promoting 
a common understanding of factors relating to healthcare 
quality across the Nordic countries. 



Although the CAHPS group has applied the CAPHS- 
questionnaire across Spanish- and English-speaking 
inpatients in the USA,^^ previous cross-national studies of 
patient experiences have not demonstrated that the 
questionnaires perform in the same manner cross- 
nationally.^'^ This makes it difficult to determine to what 
extent any differences in patient evaluations are attribut- 
able to differences in healthcare quality or questionnaire 
performance across countries. Hence, the importance of 
adequate reporting of questionnaire development and 
survey methodology has been emphasised.^ It is 

important to account for cultural and demographic 
differences, health problems and potential translation 
problems in cross-national studies of patient experi- 
ences.^ Many healthcare quality indicators have been 
studied at the local level with local providers, but national 
governments require comparisons of providers.^ 

The NORPEQ was designed to include a core set of 
questions covering the most important aspects of inpa- 
tient experiences that can be used for cross-national 
comparison of inpatient experiences alongside existing 
longer-form national survey questionnaires.^'^ This 
article describes the cross-national questionnaire devel- 
opment based on surveys undertaken among hospital 
inpatients in Finland, Norway, Sweden and Faroe Islands. 
Following a rigorous process of questionnaire develop- 
ment including forward— backwards translation, the 
NORPEQ questionnaire was assessed for levels of missing 
data, reliability and validity in the four countries. The 
development of NORPEQ followed recommended criteria 
including forward— backwards translation necessary for 
a questionnaire that is to be used cross-nationally.^ ^'^ 

METHODS 

NORPEQ development 

The appropriateness of health quality indicators for 
benchmarking and other aspects of quality improvement 
is dependent on their reliability and validity,^^ 
and criteria for their evaluation have been recom- 
mended.'^ 4 6 11 19 21 ■pj^ggg criteria were applied in the 
development of the eight-item NORPEQ which was 
translated into English by two professional translators 
(online appendix 1). Development followed a literature 
review of existing questionnaires and consultation with 
experts within the field of patient experiences in three 
face-to-face meetings. In the first of these meetings, three 
main considerations guiding the development of the 
questionnaire were decided. First, the questionnaire 
should include the most important aspects of patient 
experiences following a literature review relevant to 
patients across the Nordic countries. Second, the ques- 
tionnaire should be brief so that the questions can 
supplement existing surveys. Third, the questionnaire 
should be developed in Norwegian and translated into 
the other Nordic languages using the forward— backwards 
methodology.^'^ 

The content of questionnaires used in surveys was 
assessed for appropriateness, and patient involvement in 
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this process was designed to lend the NORPEQ content 
vahdity. Analysis of themes and items in existing 
NORPEQs revealed that the content of the question- 
naires was fairly similar/'^ but that there was some 
variation in question formulation and the choice of 
scaling. The analysis was based on the most widely used 
questionnaires in the Nordic countries/'^ which 
included the national patient experience survey in 
Denmark/^ a 20-item measure of patient experience in 
Finland,^ a short form of the Quality from the Patient's 
Perspective questionnaire on Iceland, the Patient 
Experiences Questionnaire in Norway*' and the Picker 
survey in Sweden.^" Furthermore, the review found that 
patient experiences with health personnel including 
whether the doctors were understandable, doctors' and 
nurses' professional skills, nursing care, whether the 
doctors and nurses were interested in the patients 
problems and information related to tests are the most 
important aspects of patients experiences.^'^ 

Six of the eight NORPEQ items sum to produce an 
overall scale from 0 to 100, where 100 is the best possible 
experience of care. If respondents had missing values on 
more than half of the items, mean scores were imputed. 
The NORPEQ is designed to be routinely used alongside 
longer-form instruments in Nordic and international 

9 13 18 24 

patient surveys. 

The NORPEQ was tested by means of cognitive inter- 
views with six patients. A Norwegian pilot survey of 500 
patients receiving inpatient care at a large University 
hospital found to have evidence for data quality, unidi- 
mensionality, internal and test— retest reliability and 
construct validity. ^'^ 

To ensure valid comparison across countries, ques- 
tionnaires must demonstrate cross-cultural equivalence.^ 
The NORPEQ questionnaire was developed in Norwe- 
gian for translation into the other Nordic languages 
using methods that adhere to minimum standards 
recommended for translation of patient questionnaires, 

• • 14 25 26 

including the forwards— backwards methodology. 
The questionnaire was translated into Danish, Finnish 
and Swedish by two forward translators fairly acquainted 
with the area and with some experience in health-related 
research. Emphasis was put on conceptual rather than 
literal translation. The backward translators were 
Norwegians who were not familiar with the original 
version. The Faroese version went through 
forward— backward translation from Danish to Faroese 
and back to Danish. This was because Faroese speak 
Danish, which is taught at school, and no one from the 
research group was able to translate from Norwegian to 
Faroese. Again, this included independent forward and 
backward translators to ensure that the Faroese ques- 
tionnaire was conceptually similar to the Danish version. 
The Norwegian researchers assessed the forward— back- 
wards translations following discussions with those 
responsible in the different countries, and it was agreed 
that the instructions and questions had retained their 
original meaning. 



The approved translations were then tested by means 
of cognitive interviews with six patients in Finland (n=6) 
Sweden (n=ll) and the Faroe Islands {n—27). Patients 
were asked whether they had omitted any questions, if 
any questions were difficult or too similar, if questions 
were acceptable and relevant and if they had any other 
problems with completion. 

Data collection 

In October and November 2009, 500 patients were 
randomly selected from adult inpatients at one Univer- 
sity hospital in Finland were sent a questionnaire. Postal 
reminders were sent 3 weeks after the first questionnaire. 
Five hundred patients were randomly selected from 
adult inpatients at one Swedish University hospital 
within a 3-week period in February— March 2009. Two 
postal reminders were sent to non-respondents at I and 
3 weeks. The Norwegian national survey included 24 141 
patients randomly selected among patients discharged 
from 63 hospitals in a 3-month period in 2006. About 
a week later, a random sample of 270 respondents was 
asked to complete an identical second questionnaire for 
purposes of assessing test— retest reliability. The Faroe 
Islands has only 50 000 inhabitants and hence a large 
pilot of the NORPEQ was not possible. Instead the 
NORPEQ was applied alongside a national patient 
experience survey. In May 2010, 892 inpatients were 
discharged from three different hospitals on the Faroe 
Islands, and these patients were mailed a questionnaire: 
six patients were not eligible. One postal reminder was 
sent 3 weeks later to non-respondents. 

Statistical analysis 

Questionnaire evaluation followed recommendations 
relating to measures of patient satisfaction and ques- 
tionnaire development more generally. Items were 
assessed for levels of missing data. Principal component 
analysis (PCA) with varimax rotation was used to assess 
the underlying dimensionality of the six items measuring 
patient experiences.^'^ Following previous findings, ^'^ it 
was expected that these items would be unidimensional. 
Internal consistency was assessed using item-total corre- 
lation and Cronbach's a. The former measures the 
association between the item and the remainder of its 
scale, the latter determines the overall correlation 
between items within a scale. ^'^ In the national Norwe- 
gian survey, test— retest reliability was assessed by the 
intraclass correlation coefficient. 

Construct validity assesses the extent to which a ques- 
tionnaire measures what it is intended and is assessed 
through comparisons with variables that following 
empirical and theoretical considerations have expected 
associations with patient experiences or satisfaction.^" 
Research including systematic reviews has found that 
patient experiences and satisfaction are associated 
with general satisfaction, perceptions of incorrect treat- 
ment,*^ ^'^ health status, health outcomes, fulfilment of 
expectations.^ ^'^ We used responses to five 
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additional items to assess construct validity of the 
NORPEQ scores including the two items relating to 
general satisfaction and incorrect treatment that have 
been widely used in Nordic and international research 
and items relating to general health,^ ^'^ ^"^ changes in 
general health and physical health compared with 
before admission, and fulfilment of expectations. These 
items all have 5-point descriptive scales. For an overview 
of stages in the development and evaluation of the 
NORPEQ instrument see Table 1. 

RESULTS 

NORPEQ development 

Only minor changes were made to the questionnaire on 
the basis of the forward— backward translations. The 
results of cognitive interviews showed that the eight items 
were acceptable, relevant and understandable to Norwe- 
gian and Swedish inpatients. One challenge related to the 
use of the term 'health personnel', which in Norwegian 
includes nurses and licensed practice nurses, whereas in 
other countries, nurses comprise other groups or they do 
not have a corresponding concept. While there were no 
problems for the other patient groups, some Finnish 
patients found the item about whether doctors and health 
personnel were interested in patient's problem difficult 
because it asked about two occupational groups. However, 
the majority of informants in Finland did not rate the 
question as difficult, and most found it relevant with 
appropriate response categories. Therefore, no changes 
to the NORPEQ were made following cognitive interviews 
in Finland, Norway and Sweden. Several patients from the 
Faroe Islands found the question relating to incorrect 
treatment difficult to understand, and hence, the ques- 
tion was reformulated. 

Data collection 

Table 2 shows the response rate and respondent and 
non-respondent characteristics for each country. Of 496 
eligible patients, 383 (77.2%) responded to the Finnish 
survey, and in the Swedish survey, 412 (84%) patients 
responded. Of the 24 141 patients included in the 
Norwegian national survey, 11079 (45.8%) responded. 
For the Faroe Islands, 551 (62.2%) responded. 



Table 1 Stages in the development and evaluation of tlie 


NORPEQ 




1 Item derivation 


Literature review, 




Nordic surveys, 




expert consensus 


2 Translation 


Forwards— bacl<wards 


3 Patient interviews 


Cognitive testing 


4 Pilot survey 


Data quality— missing data, 




response distribution 


5 Main survey 


Dimensionality, internal 


u 


consistency, test— retest 




reliability, construct validity 


NORPEQ, Nordic Patient Experiences Questionnaire. 



In Finland, the respondents' mean age was 59.1 
(SD=I7.6) years and 51.7% were women. Respondents 
were on average approximately 9 years older than non- 
respondents, which was statistically significant. Respon- 
dents in Sweden were on average approximately 8 years 
older, which was statistically significant. In the Norwe- 
gian national stirvey, the respondents' mean age was 
60.35 (SD 16.97) years and 53.2% were women (table 2). 
There were a significantly slightly greater proportion of 
female respondents. Compared with non-respondents, 
respondents were also significantly more likely to be 
admitted in an emergency and have 0.47 fewer days in 
hospital. For the Faroe Islands, the respondents' mean 
age was 57.0 (23.6) years and 47.7% were women. 

Statistical analysis 

Missing data for the six NORPEQ items ranged from 
0.5% to 3.9% across the three smaller surveys in Finland, 
Norway and Sweden (table 2). For the Faroe Islands, the 
level of missing data varied between 6.7% and 10%. 
Incorrect treatment had the highest level of missing data 
for all surveys, the only exception was Norway where 
general satisfaction for the national survey had the 
highest level of missing data. Item means for the four 
surveys were generally skewed towards positive experi- 
ences of care ranging from 3.7 to 4.4 on the 1—5 scale. 
The mean ranking of items was very similar across the 
surveys. Patients had the poorest experiences on the two 
items information relating to tests or health personnel 
being interested. Patients had the best experiences with 
nursing care. 

Results of PGA and measures of internal consistency are 
shown in table 3. The Finnish data gave two components, 
with eigenvalues of 3.38 and 1.02. One component was 
found for the Swedish, Norwegian national and for the 
Faroese data with eigenvalues of 3.82, 3.68 and 3.62, 
respectively. Given the relatively low value for the second 
component found for the Finnish data that just meets the 
criterion of 1.0, a single component that comprised all six 
items was accepted. The six items had high component 
loadings between 0.70 and 0.82 on this single component. 
The component loadings had a similar pattern across the 
different countries with the items 'health personnel 
interested in health problem' and 'information on tests' 
generally having the highest loadings and 'nursing care' 
and 'doctors understandable' generally having the lowest 
loadings. Item-total correlations for the six NORPEQ 
items were acceptable and ranged from 0.53 to 0.80. 
Cronbach's a for the six NORPEQ items ranged 
from 0.84 to 0.88 for Finland and the Faroe Islands, 
respectively. 

Of 270 randomly selected Norwegian patients asked to 
take part in the test— retest survey in the national 
Norwegian survey, 196 (72.6%) returned a second 
questionnaire. The intraclass correlations ranged from 
0.68 to 0.73 for the items relating to information on test 
and nurses' professional skills, respectively. The intra- 
class correlation for the NORPEQ scores was 0.85, which 
is considered acceptable for group comparisons.'"* 
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Table 3 Component loadings and internal consistency of the NORPEQ 












National 










m 


Finland 




Norway 




Norway 




Sweden 




Faroe 






Comp 


CA/item 


Comp 


CA/item 


Comp 


CA/item 


Comp 


CA/item 


Comp 


C/\/item 




load 


total 


load 


total 


load 


total 


load 


total 


load 


total 


NORPEQ score 




0.84 




0.85 




0.87 




0.88 




0.87 


Doctors understandable 


0.74 


0.62 


0.71 


0.59 


0.76 


0.65 


0.80 


0.71 


0.75 


0.64 


Doctors' 


U.Jo 


O.bo 


O./o 


U.DO 


0.78 


0.68 


U./O 
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Uoo A 


professional skills 






















Nurses' 


0.70 


0.56 
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0.79 


0.68 


n 7Q 


u.oy 


U./O 


U.OO 


professional skills 






















Nursing care 


0.68 


0.53 


0.73 


0.60 


0.77 


0.65 


0.77 


0.67 


0.75 


0.63 ^ 


Health personnel 


0.82 


0.71 


0.81 


0.71 


0.83 


0.74 


0.87 


0.80 


0.81 


0.72 . 


interested in problem 






















Information on tests 


0.80 


0.69 


0.78 


0.66 


0.76 


0.65 


0.80 


0.70 


0.79 


0.69 


NORPEQ, Nordic Patient Experiences Questionnaire. 







































Table 4 shows that the NORPEQ scores for the four 
countries were skewed towards positive experiences and 
varied from 75.3 to 78.6 for Finland and Sweden, 
respectively. Approximately 10% of the patients in both 
of the Norwegian and the Swedish surveys scored 60 or 
below. In Finland, 15.7% of Finnish patients scored 58.3 
or below, while on the Faroe Islands, 16.3% scored 58.3 
or below. Scores for the six items were also similar with 
all three countries scoring lowest on the item relating to 
information on test and examinations. Finland, Sweden 
and the Faroe Islands had the highest scores on the item 
relating to nursing care, while Norway had very similar 
high scores for this item and two others. Scores for the 
satisfaction item were also generally high. 

Table 5 shows the results of validity testing. The 
correlations across the different surveys are broadly 
consistent. High levels of correlations were found 
between the NORPEQ scores and general satisfaction in 
the range 0.72—0.77 and correlations with incorrect 
treatment were lower and in the range 0.24—0.39 for all 
surveys. For Finland, Norway and Sweden, correlations 
with expectations relating to treatment and care were 
moderate and in the range 0.51—0.58. Correlations with 
expectations relating to health outcome were lower and 
in the range 0.30—0.38. Lower levels of correlation were 
found for the health-related variables, for example, 
general health had low correlations with NORPEQ 
scores in Norway and Sweden, whereas in Finland, 
general health did not correlate with NORPEQ. The 
health-related variables were not available for the Faroe 
Islands. 

DISCUSSION AND CONCLUSIONS 
Discussion 

The NORPEQ is a short self-completed questionnaire 
with evidence for data quality, reliability and validity. The 
NORPEQ includes what are judged to be the most 
important aspects of experiences for patients.^ Levels of 
missing data were very low across the eight items for 
Finland, Norway and Sweden. The results of PCA showed 



one uniform measure of patient experiences based on 
the six NORPEQ items, and general satisfaction and 
incorrect treatment are treated as supplementary items. 
Overall, this shows that NORPEQ meets the objective to 
develop a short questionnaire covering the most 
important aspects of patient experiences within the 
Nordic countries. 

Furthermore, the NORPEQ shows evidence for 
construct validity in tests that were based on hypothesised 
associations with variables relating to general satisfaction, 
incorrect treatment,^ ^'^ health status, health outcomes 
and fulfilment of expectations.^ ^ ^"^ However, 
although it was negative which followed the hypothesis, 
the correlation between general health and the 
NORPEQ for the Finnish data was close to zero. The 
future inclusion of more specific questions that relate to 
different aspects of health will help determine whether 
this finding has implications for cross-cultural validity. 

The test— retest reliability of the NORPEQ has been 
found to be acceptable in two Norwegian surveys at the 
local and national levels. It would be an advantage to 
have test— retest data for the other countries as well. 
Previous studies have found comparable results for 
internal consistency and test— retest reliability for 
measures of patient experiences.' Therefore, it is 
reasonable to assume similar levels of test— retest reli- 
ability for the other countries given the similarity of the 
results across the countries. 

Compared with other surveys of patient experiences at 
a national level, the samples for the surveys were small 
and three of them were pilot studies. However, the 
different analyses still produced satisfactory results and 
items performed very similarly across the countries. 
Mean item scores, component loadings and item-total 
correlations were similar across countries. Results of 
validity testing were in line with those following national 
surveys that have used longer questionnaires comprising 
scales relating to different aspects of patient experiences 
and satisfaction." NORPEQ scores had similar signifi- 
cant levels of correlation with those for the additional 
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Table 5 Correlation of NORPEQ scores with patient perceptions of satisfaction, incorrect treatment, expectations, liealth and 
outcomes 





Finland 


Norwayt 


National Norway 


Sweden 


Faroe Islands 


General satisfaction 


0.77** 


0.74** 


0.72** 


0.73** 


0.73** - 


Incorrect treatment 


-0.35** 


-0.39** 


-0.39** 


-0.24** 


-0.31** M 


Expectations: treatment and care 


0.53** 


0.51** 


0.57** -If. 


0.58** 




Expectations: outcomes 


0.30** 


0.32** 


0.37** 


0.38** 




General healtli 


-0.09 


-0.19** 


-0.24** 


-0.24** 




General heaitii compared with before admission 




0.22* 


0.27** 






Physical health compared with before admission 




0.27** 


0.22** 







Statistical significance:*p<0.05; ** p<0.01. 
fThese results were reported in a previous article. 

tThis question has a slightly more general wording in the national Norwegian survey than in the other pilots.' 
NORPEQ, Nordic Patient Experiences Questionnaire. 



items used in validity testing, the exception being 
general health for Finland. The strong associations 
between patient experiences as measured by NORPEQ 
and general satisfaction in each country confirm the 
importance of the NORPEQ items as a measure of 
patient experiences across countries. 

The measurement of patient experiences is an 
important component of patients' evaluation of health- 
care services.^ During the past decade, measures of 
patient experiences have gained increasing interest as 
healthcare quality indicators. Most studies of healthcare 
quality indicators have taken place at the local level, and 
few studies have conducted comparisons across coun- 
tries.^ It is thus difficult to know whether variations in 
patient experiences reflect differences in culture, health 
systems or actual differences in quality as perceived by 
patients. 

Methodological concerns aside, existing research has 
implied that there is cross-national variation in patient 
experiences.'^^ The Commonwealth Fund and the WHO 
have conducted surveys of the general population'^^ 
and within primary care.'^* Recipients of the survey 
results have included health ministers and decision- 
makers in each of the countries. Commonwealth Fund 
surveys include a large number of single items, and 
there has been no reporting of data quality, reliability 
and validity.^ The Picker Institute has compared 
patients' perceptions of the quality of acute hospital 
care as well as more focused surveys of patient experi- 
ences across Germany, Sweden, Switzerland, the UK and 
the USA.^^ These studies have focused on results of 
surveys and have not described the translation processes 
and cross-cultural testing.^ The Consumer Assessment 
of Health Plan Surveys has published the results of one 
cross-cultural evaluation of its questionnaires,'^'' and it is 
translated for use in the Netherlands.^'' However, 
Consumer Assessment of Health Plan Surveys ques- 
tionnaires have not been used in international 
comparisons.^ Finally, one study used existing data 
for 12 European countries in comparisons of the quality 
of primary care.'^^ However, evidence for the cross- 
cultural equivalence of the included instruments was 
not given. 



The present study is the first to report on the necessary 
evaluative work to support cross-national comparisons of 
patient experiences for several countries. One study 
included a cross-national comparison of public trust in 
healthcare in Germany, the Netherlands and England 
and Wales that has followed guidelines for questionnaire 
development including forward— backward translation 
and cross-cultural testing. In contrast to the similar 
ranking of patients' confidence in both nurses' and 
doctors' skills across the NORPEQ countries, this study 
revealed significant differences across the countries 
when comparing public trust in healthcare providers 
including doctors. '^'^ However, patient experiences and 
public trust are different constructs, and the two surveys 
of these constructs were undertaken in different popu- 
lations. The NORPEQ is the first questionnaire devel- 
oped to compare general hospital inpatients' 
experiences across countries, which is the most studied 
and largest group of healthcare users. 

Furthermore, patient experiences were assessed in 
four countries with highly similar healthcare delivery. 
This similarity may have contributed to the consensus in 
identifying important aspects of experiences for inclu- 
sion in the NORPEQ. The process of item development 
included experts from the different countries who 
subsequently monitored all phases of the project. The 
translation process followed recommended procedures 
relating to cross-cultural adaptation within health- 
related research. 

The study included two national surveys for Norway 
and the Faro Islands, but two surveys of just one 
University hospital for two other countries, Finland and 
Sweden, which limits the conclusions that can be drawn 
in terms of the appropriateness of the NORPEQ and 
cross-cultural equivalence. However, the national survey 
results for Norway and the Faroe Islands were very 
similar to those reported in an earlier study of the 
NORPEQ that included patients from one Norwegian 
University hospital using the similar survey design that 
was reported here for Finland and Sweden. Hence, it is 
reasonable to hypothesise that the NORPEQ will 
perform similarly in national surveys within these coun- 
tries and in other Nordic countries following 
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recommended translation procedures and cognitive 
testing. To further assess the NORPEQ for cross-cultural 
equivalence, evaluation following national surveys based 
on representative samples including the other Nordic 
countries should be conducted. Moreover, the inclusion 
of NORPEQ alongside existing instruments used in 
national surveys of inpatient experiences or other 
patient groups' experiences will further contribute to 
the cross-national evaluation of validity. 

Practice implications 

This study has demonstrated that it is possible to 
cover important aspects of the healthcare experiences 
in a short-form measure. Furthermore, the 8-item 
questionnaire has proved relatively easy to translate 
and evaluate across the Nordic countries. Through 
the inclusion of the NORPEQ in national surveys of 
Norway and the Faroe Islands, the current study has 
shown how the brevity of the questionnaire lends it 
feasibility for inclusion as part of existing national 
surveys. Hence, we consider the NORPEQ suitable as 
a supplement to future national healthcare surveys 
conducted in the Nordic countries that can facilitate 
cross-national comparisons to inform Nordic citizens, 
patients, politicians and leaders and personnel in 
healthcare about patient experiences as a measure of 
the quality of healthcare. This will promote a mutual 
understanding of quality in healthcare across the 
Nordic countries. 

Conclusions 

Although NORPEQ is brief, it measures important 
aspects of patient experiences directly related to the care 
patients receive in the hospital. Items typically used in 
longer questionnaires such as hospital access, hospital 
equipment or standards of patient facilities*' are not 
included. Still, results of testing for validity including the 
high level of correlation with responses to the general 
satisfaction item follow previous findings for longer 
questionnaires.^ The strength of NORPEQ is the focus 
on important aspects of healthcare summed to form 
a single score that is, supplemented with two items 
relating to patient perceptions of incorrect treatment 
and satisfaction. The NORPEQ is acceptable and feasible 
for cross-national comparisons of hospital care but 
should also be considered for applications alongside 
existing national surveys for the Nordic countries. 
Overall, the NORPEQ scores show that there is consid- 
erable scope for improvement in delivery of care in the 
Nordic countries. 

To conclude, the NORPEQ has acceptable data quality, 
reliability and validity within the Faroese, Finnish, 
Norwegian and Swedish inpatients. The questionnaire 
has been translated into other Nordic languages, 
cognitive interviews have taken place in Denmark and 
further testing is planned in Iceland. Following further 
evaluation, the NORPEQ will be used in future national 
and cross-national surveys. The NORPEQ was developed 



and tested as part of a Nordic quality indicator project 
that aims to assess and compare health service perfor- 
mance across the Nordic countries. Such cross-national 
comparisons of health system performance offer greater 
accountability and transparency, support strategy review 
and development, and give potential for mutual 
learning. 
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